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Abstract 

This  study  complements  research  indicating  that  content  area  novices  judge  importance 
in  texts  according  to  sentence  type  category  (eg.,  whether  sentences  are  definitions,  facts, 
equations,  etc.).  Subjects  varying  in  expertise  judged  the  importance  of  sentences  in 
physics  texts  when  they  were  presented  in  one  of  two  forms:  definitions  or  facts 

(Experiment  1),  and  equations  or  verbal  formulae  (Experiment  2).  The  two  sentence  versions 
were  always  identical  in  substantive  content.  Experts  and  subjects  without  physics  training 
judged  these  variants  similar  or  equal  in  importance.  However,  beginning  physics  students 
judged  definition  and  equation  versions  as  more  important.  Thus  sentence  form  is  a  salient 
text  feature  for  beginning-level  students,  who  have  developed  general  rules  about  what 
categories  of  information  are  important  in  physics.  Sentence  category  is  irrelevant  for 

experts,  who  have  rich  content  schemas  allowing  them  to  judge  importance  directly. 

Sentence  category  also  has  little  effect  on  people  without  physics  training,  who  lack  strong 
expectations  regarding  what  types  of  information  are  important.  These  results  have 
theoretical  implications  for  understanding  content  schema  development,  and  practical 

implications  for  textbook  writers.  ^  ' 
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Novice  Importance  Rules:  Definitions  and  Equations 

Diana  Dee-Lucas  &  Jill  H.  Larkin 
Carnegie-Mellon  University 

Scientific  textbooks  are  typically  densely  packed  with  complex  information,  including 

equations,  symbols,  and  specialized  terms  Consequently,  it  can  be  very  difficult  for 

students  who  are  unfamiliar  with  scientific  subject  matter  to  distinguish  the  important  content 
from  the  elaborative  information  when  reading  this  type  of  text.  The  purpose  of  the  present 
research  was  to  investigate  rules  used  by  novice  readers  (i.e. ,  readers  who  are  unfamiliar 
with  the  text  content  domain)  in  determining  what  is  important  in  scientific  texts.  The 

studies  reported  here  tested  the  hypothesis  that  novices  judge  importance  on  the  basis  of 
category-membership  rules  --  that  is,  that  they  consider  certain  information  to  be  important 
simply  because  of  its  category  membership  (i.e.,  whether  it  is  a  definition,  equation,  fact, 

etc.),  regardless  of  its  content.  This  was  done  by  using  minor  wording  changes  to  vary  the 

category  membership  of  selected  information  in  physics  texts,  and  examining  how  the 

category  changes  influenced  experts'  and  novices'  judgements  of  the  importance  of  the 
content.  Two  experiments  contrasted  the  judged  importance  of  identical  information  when  it 
was  presented  as  a  definition  or  a  fact,  and  when  it  was  slated  as  an  equation  or  written 
out  in  sentence  form. 

There  are  various  sources  of  information  other  than  category-membership  rules  that 
novice  readers  could  use  in  assessing  importance.  Most  research  has  focused  on  how  text- 
based  indicators  of  importance  such  as  text  structure  and  signaling  devices  (eg., 

imriorlinino  adjunct  questions,  stngino  tvpnqmphionl  rninn  ntn  )  influence  mndinn  Thor;.' 
textual  manipulations  are  "content-free."  in  that  their  effects  should  not  depend  on  the 

nature  of  the  text  content  or  the  expertise  of  the  reader.  In  contrast,  a  "content-specific" 
source  of  information  for  assessing  importance  in  texts  is  the  reader’s  "content  schema" 
(Kieras.  1985).  A  content  schema  consists  of  domain-specific  knowledge  about  how 
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information  in  a  content  area  is  typically  organized,  including  what  is  important. 

Past  research  has  largely  ignored  the  role  of  content-based  schemas  in  the 
comprehension  of  novice  readers.  This  is  because  it  has  typically  been  assumed  that 
readers  who  are  unfamiliar  with  the  content  domain  of  a  text  would  lack  such  a  schema 
(see  Kieras,  1985,  for  discussion).  However,  recent  work  by  Dee-Lucas  and  Larkin  (1986) 
suggests  that  novices  do  develop  a  rudimentary  "content  schema"  for  scientific  content 
domains.  This  content  schema  consists  of  rules  specifying  what  types  of  easily-recognizable 
information  (i.e. ,  definitions,  facts,  equations,  etc.)  are  important  in  physics  texts.  The 
current  study  examined  the  basis  on  which  these  rules  are  formed. 

Novice  importance  rules  have  been  investigated  in  research  comparing  the  importance 
judgements  of  expert  and  novice  physicists  for  different  types  of  information  in  physics  texts 
(Dee-Lucas  &  Larkin,  1986).  Although  this  research  found  that  experts  and  novices  generally 
agreed  on  the  relative  importance  of  various  types  of  information,  the  novices  did  not 
distinguish  between  the  important  and  unimportant  content  within  type  categories.  For 
example,  both  groups  judged  definitions  to  be  more  important  than  facts,  but  novices  were 
even  more  likely  than  experts  to  judge  definitions  as  important  and  facts  as  unimportant. 
This  suggests  that  the  novices  had  formulated  a  general  rule  that  definitions  are  more 
important  than  facts  and  judged  importance  on  this  basis,  without  discriminating  between 
important  and  unimportant  definitions  and  facts.  Unlike  novices,  the  experts'  importance 
judgements  were  not  tied  as  closely  to  category  membership.  They  presumably  were 
judging  importance  on  the  basis  of  the  nature  of  the  specific  sentence  content,  rather  than 
the  type  category. 

The  findings  from  this  area  of  research  suggest  that  people  just  beginning  to  learn 
about  physics  develop  a  set  of  rules  defining  what  types  of  information  are  important  in  that 
domain.  They  use  these  rules  both  in  deciding  what  is  important  (Dee-Lucas  &  Larkin  1986) 
and  in  guiding  attention  during  reading  (Dee-Lucas  &  Larkin.  1987).  However,  this  research 
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on  novice  importance  rules  has  not  controlled  for  content  differences  between  information 
categories.  Therefore,  the  basis  used  by  novices  in  deciding  that  one  category  is  more 
important  than  another  is  not  clear.  Novices  in  these  studies  may  have  been  basing  their 
importance  judgements  on  some  feature  of  the  content  that  differed  between  the  categories, 
rather  than  on  the  categories  themselves.  For  example,  novices  may  have  considered 
definitions  to  be  more  important  than  facts  because  the  definitions  contained  more  new 
terms.  If  this  were  the  case,  then  the  rule  used  by  the  novices  would  be  that  statements 
with  new  terms  are  more  important  than  statements  involving  known  terms,  rather  than  a 
rule  that  definitions  are  more  important  than  facts.  Thus  it  is  not  known  whether  novices 
are  judging  importance  according  to  a  superficial  analysis  of  category  membership,  or  a 
"deeper"  analysis  of  the  nature  of  the  information  typically  contained  in  various  categories. 
A  more  precise  understanding  of  the  basis  for  novice  importance  rules  would  suggest  how 
domain-specific  content  schema  evolve  in  novice  learners.  It  would  also  have  implications 
for  how  text  writers  can  effectively  guide  attention  to  important  content  in  texts  geared  to  a 
novice  audience. 

The  current  study  examined  the  basis  for  novice  importance  rules.  It  specifically 
tested  the  hypothesis  that  novice  importance  rules  are  based  solely  on  category  membership; 
in  other  words,  that  novices  consider  certain  information  to  be  important  simply  because  of 
its  category  membership,  regardless  of  content.  This  was  done  by  varying  the  category 
membership  of  specific  information  in  physics  texts,  and  examining  how  the  category  change 
influenced  experts'  and  novices'  judgements  of  the  importance  of  the  content.  If  novice 
importance  rules  are  based  on  category  membership,  then  novices  should  vary  their 
assessment  of  the  importance  of  a  given  statement  with  changes  in  its  category.  Experts, 
on  the  other  hand,  should  be  relatively  uninfluenced  by  category  changes  because  they 
would  presumably  judge  importance  on  the  basis  of  the  nature  of  the  content. 

The  category  membership  of  sentence  information  was  manipulated  through  minor 
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wording  variations  which  did  not  alter  the  primary  sentence  content,  but  signalled  the  content 

i 

as  belonging  to  a  particular  category.  In  this  way.  category  membership  could  be  varied  by 
changing  the  form  in  which  the  information  was  presented  while  content  was  held  constant. 

Thus  unlike  earlier  studies,  this  research  assessed  the  effects  of  category  membership  on  i 

perceived  importance  independent  of  category  content. 

Two  studies  were  conducted  in  which  expert  and  novice  physicists  judged  the 
importance  of  information  in  physics  texts  Experiment  1  compared  the  judged  importance 
of  attributive  information  when  it  was  presented  in  the  form  of  a  definition  or  a  fact.  The 
purpose  was  to  determine  whether  differences  in  the  perceived  importance  of  definitions  and 
facts  found  in  earlier  research  were  due  to  differences  in  category  membership  as  opposed 
to  content  differences.  This  was  determined  by  contrasting  the  judged  importance  of  i 

information  when  it  was  signalled  as  being  a  definition,  through  the  use  of  the  words  "is  / 

defined  as",  and  when  this  signalling  was  absent  or  replaced  with  a  neutral  phrase  so  that  ! 

the  content  appeared  to  be  a  fact.  In  this  way  the  judged  importance  of  definitions  and  ; 

facts  was  compared  while  holding  sentence  content  constant  across  categories. 

Experiment  2  extended  the  findings  from  the  first  study  to  a  class  of  information  which  • 

is  particularly  important  in  physics  problem-solving  -  quantitative  relations.  It  contrasted  the  * 

judged  importance  of  quantitative  relations  when  they  were  presented  as  equations  (eg.,  a  1 

=  b/c)  or  written  out  in  verbal  form  (e  g  ,  a  is  equal  to  b  divided  by  c).  The  equational  j 

I 

form  signals  the  content  as  being  quantitative  in  nature,  while  the  verbal  form  makes  the 

category  membership  of  the  content  less  apparent.  Previous  research  indicates  that  novices 

(as  well  as  experts)  consider  equations  to  be  a  particularly  important  type  of  content  in  ! 

physics  texts  (Dee-Lucas  &  Larkin  1986).  If  novices  are  basing  this  judgement  on  a 

category-membership  rule,  (i.e  .  that  equations  are  important  regardless  of  content)  then  they  ' 

would  judge  the  same  quantitative  information  as  more  important  when  it  was  presented  as 

an  equation  rather  than  in  verbal  form.  j 
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In  each  study,  two  physics  passages  were  used  which  contained  target  sentences  that 

|  could  be  expressed  in  different  forms  There  were  two  versions  of  each  passage,  each 

version  being  identical  except  for  the  form  in  which  the  target  sentences  were  presented. 

> 

|  The  importance  judgements  of  the  novices  and  experts  for  the  fact  and  definition  versions 

k 

j  (Experiment  1)  and  the  equation  and  verbal  versions  (Experiment  2)  of  the  target  sentences 

•  were  compared  to  see  to  what  extent  the  sentence  form  influenced  the  perceived  importance 
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of  the  information. 


Experiment  1 

The  goal  of  this  experiment  was  to  determine  if  novices  consider  definitions  to  be 
more  important  than  facts  independent  of  the  content  of  the  two  sentence  types. 

Method 

Stimulus  Materials.  One  passage  was  about  work  and  energy  and  one  dealt  with  fluid 
statics.  Each  was  about  50  sentences  long.  One  contained  9  target  sentences  and  one 
had  11  target  sentences. 

The  definition  and  fact  versions  of  the  target  sentences  differed  in  that  definitions 
always  included  "is  defined  as."  and  thus  were  signaled  as  being  definitions.  In  the  fact 
versions,  "is  defined  as"  was  dropped  or  replaced  with  "is  represented  as,"  "is  calculated 
as."  or  "is  indicated  by."  Thus  the  facts  were  "non-definitions"  in  that  in  place  of 
definition  signaling  they  contained  phrases  indicating  that  the  sentence  was  presenting 
attributive  information  about  the  sentence  topic  (as  opposed  to  criterial  attributes  defininq  the 
sentence  topic). 

Examples  of  the  definition  and  fact  versions  of  some  of  the  target  sentences  are 
shown  in  Table  la  There  were  two  versions  of  each  passage.  In  one  version,  the  odd- 
numbered  target  sentences  were  definitions  and  the  even-numbered  were  facts:  in  the 


second  version  this  was  reversed 
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Insert  Table  1  about  here 

Each  of  the  target  sentences  was  classified  according  to  its  level  in  the  hierarchical 
structure  of  the  passage.  The  procedure  used  for  the  structural  analysis  is  reported  in  Dee- 
Lucas  and  Larkin  (1986).  This  analysis  produced  a  hierarchy  with  the  main  topics  or 
concepts  occurring  at  the  highest  levels  and  modifying  information  occurring  at  the  lower 
levels.  Modifying  information  consisted  of  examples,  attributes  and  properties,  derivations 
(i.e..  information  implied  by  or  derived  from  higher  level  information),  explanations,  sub-topics, 

and  preconditions  (i.e..  necessary  conditions  for  a  rule,  principle,  or  fact  to  hold  true).  The 

hierarchical  analysis  was  performed  at  the  sentence  level.  There  were  7  sentences  at  level 
1  (the  most  superordinate  level).  6  sentences  at  level  2.  and  7  sentences  at  level  3. 
Hierarchical  level  was  included  as  a  variable  in  the  data  analyses  to  see  whether  perceived 
importance  was  influenced  by  level,  and  if  this  variable  interacted  with  sentence  form  (i.e.. 
definition  or  fact). 

Subjects  The  novices  were  24  undergraduates  with  2  or  3  semesters  of  college 

physics.  Novices  with  this  level  of  physics  training  were  selected  to  insure  that  the  novice 
group  had  had  enough  exposure  to  physics  to  have  developed  information-type  rules,  but 
had  not  approached  the  expert  level  in  training.  The  24  experts  had  completed  at  least 

one  year  of  graduate  study  in  physics. 

Two  control  groups  were  also  run  in  the  experiment  to  see  if  expert-novice  differences 

in  tho  porroivpd  importance?  pf  Ihp  \\mrn  rfijn  fr*  rfiff  in  nHw'-'n»i'-'nnl 

level  (i.e..  undergraduate  vs.  graduate  level  training)  as  opposed  to  differences  in  physics 
knowledge.  The  two  control  groups  were  selected  so  as  to  differ  <n  their  educational  level 
in  the  same  manner  as  the  two  experimental  groups  However,  none  of  the  control  group 
subjects  had  taken  any  college-level  physics,  so  they  were  similar  to  each  other  in  terms  of 
their  physics  knowledge.  The  undergraduate  control  group  consisted  of  24  undergraduates. 
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the  graduate  student  control  group  consisted  of  people  who  had  completed  at  least  1  year 
of  graduate  training  in  the  humanities  or  social  sciences.  Although  this  group  will  be 
referred  to  as  the  graduate  student  control,  it  included  some  post-doctoral  researchers  and 
faculty.  This  was  also  true  of  the  corresponding  expert  experimental  group. 

Although  the  control  groups  were  specifically  selected  to  control  for  educational 
differences,  they  would  also  indicate  differences  in  perceived  importance  due  to  age. 
maturity,  and  verbal  ability.  In  the  case  of  verbal  ability,  it  is  reasonable  to  assume  that 
graduate  students  in  the  social  sciences/humanities  would  be  as  high  in  verbal  ability  as 
graduate  students  in  physics.  Similarly,  it  is  likely  that  undergraduates  attending  the  same 
university  are  roughly  equivalent  in  verbal  ability. 

Procedure  The  subjects  were  given  one  version  of  each  passage.  They  were  told  to 
read  each  passage  carefully,  then  rate  the  importance  of  each  sentence  on  a  scale  from  1 
(most  important)  to  5  (least  important),  and  then  indicate  the  10  most  important  sentences  in 
each  passage.  The  instructions  for  the  rating  task  indicated  that  each  rating  should  be 
used  at  least  once.  All  of  the  sentences  were  rated,  but  only  the  ratings  for  the  target 
sentences  were  analyzed.  The  order  in  which  the  passages  were  read  and  the  versions  of 
the  passages  received  were  counterbalanced 

The  novice  and  the  undergraduate  control  groups  were  told  that  in  completing  the 
tasks,  they  were  to  indicate  which  sentences  they  thought  would  be  most  important  to  learn 
if  they  were  going  to  be  tested  on  the  passage  content.  The  expert  and  the  graduate 
student  control  groups  were  told  to  pretend  that  they  were  teaching  a  course  and  indicate 
which  sentences  they  thought  were  most  important  for  their  students  to  learn.  These 
instructions  match  those  used  by  Dee-Lucas  and  Larkin  (1986)  in  their  initial  research  on 
expert-novice  differences  in  perceived  importance  The  instructions  were  designed  to 
compare  what  novices  think  they  should  learn  with  what  experts  (their  instructors)  think 
novices  should  learn. 
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Results 

The  data  from  the  two  dependent  measures  were  analyzed  in  two  ways.  The  ratings 
data  were  analyzed  using  a  multiway  frequency  analysis.  This  analysis  fits  a  loglinear  model 
to  categorical  data.  The  number  of  responses  in  each  rating  category  (1  through  5)  for 
each  sentence  type  (definition  and  fact)  occurring  at  each  level  (1  through  3)  was  tabulated 
for  each  of  the  subject  groups  The  multiway  frequency  analysis  was  performed  on  the  total 
number  of  responses  occurring  in  each  of  these  cells. 

The  data  from  the  sentence  selection  task  (i.e. ,  select  the  10  most  important 
sentences)  were  analyzed  with  a  logistic  regression.  The  variables  entered  in  the  analysis 
for  each  target  sentence  for  each  subject  were  sentence  type  (definition  or  fact),  level  (1 
through  3).  and  subject  group  (novice  or  expert).  The  dependent  measure  was  whether  or 
not  the  sentence  had  been  selected  as  one  of  the  most  important. 

The  data  from  the  two  control  groups  were  submitted  to  identical  analyses.  The 
results  of  these  analyses  were  compared  to  the  results  of  the  corresponding  analyses  of  the 
experimental  group  data  to  determine  if  expert-novice  differences  were  also  reflected  in 
differences  between  graduate  students  and  undergraduates  who  had  had  no  advanced 
physics  training. 

Ratings  data:  Experimental  groups.  The  multiway  frequency  analysis  of  the  ratings 
data  indicated  that  the  best-fitting  model  was  a  hierarchical  model  including  the  type  x 
group  interaction  and  the  main  effect  of  level  (A2  =  23  80,  dt=  30,  pc. 78).  The  mean  ratings 
predicted  by  the  model  for  the  type  x  group  interaction  are  shown  in  Figure  la  The 
predicted  means  for  the  novices  are  1.67  when  the  sentence  was  in  the  form  of  a  definitin" 
and  1.89  when  it  was  in  the  form  of  a  fact.  For  experts,  the  predicted  ratings  are  1  79  for 
definitions  and  1.82  for  facts.  This  interaction  indicates  that  novices  were  influenced  in  their 
importance  ratings  by  the  form  in  which  the  information  was  presented.  They  considered 
the  same  content  to  be  more  important  if  it  was  stated  as  a  definition  as  opposed  to  a 
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fact.  The  experts,  on  the  other  hand,  did  not  appear  to  base  their  ratings  on  sentence 
form;  there  is  very  little  difference  in  their  predicted  mean  ratings  for  definition  and  fact 
versions  of  the  target  sentences. 

Insert  Figure  1  about  here  ! 
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The  parameter  estimates  for  the  main  effects  and  interaction  for  the  complete  model 
are  shown  in  Table  2.  Because  of  the  usual  constraints  placed  on  the  model,  all  parameter 
estimates  for  each  main  effect  and  interaction  are  constrained  to  sum  to  zero.  Therefore, 
for  all  effects  the  magnitude  of  the  parameter  estimates  for  each  variable  are  the  same  but 
in  the  opposite  direction.  The  ratios  of  the  estimates  to  the  standard  errors  indicate  the 
degree  to  which  the  parameter  estimates  differ  from  zero. 

Insert  Table  2  about  here 


The  parameter  estimates  for  the  main  effect  of  group  show  little  difference  between 
the  experts  and  novices  in  their  use  of  the  five  rating  categories.  The  largest  differences 

occurred  in  the  use  of  rating  5  (the  lowest  rating)  and  rating  3  (the  middle  rating).  The 

novices  tended  to  use  the  rating  3  category  more  often  than  the  experts  (as  indicated  by 
the  positive  parameter  estimate),  while  the  experts  tended  to  use  the  lowest  rating  more 

often  than  the  novices.  This  suggests  that  the  experts  rated  the  target  sentences  lower  in 
importance  than  the  novices. 

The  parameter  estimates  for  the  main  effect  of  type  indie-1*"  that  the  largest  difference 
occurred  in  the  rating  3  category  (the  middle  rating).  The  negative  parameter  estimate 

indicates  that  this  rating  was  used  more  often  with  facts  than  definitions  There  were  also 


smaller  differences  in  the  use  of  the  first  two  rating  categories,  with  the  definitions  rated  1 
or  2  more  often  relative  to  facts.  This  indicates  that  definitions  were  rated  higher  overall  in 
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importance  than  facts. 


The  type  x  group  interaction  estimates  indicate  that  the  greatest  differences  between 
experts  and  novices  in  rating  definitions  and  facts  occurred  in  the  first  two  rating  categories. 
The  novices  were  more  likely  than  experts  to  give  a  target  sentence  a  rating  of  1  if  it  was 
in  the  form  of  a  definition,  and  somewhat  more  likely  to  rate  it  a  2  if  it  was  in  the  form  of 
a  fact.  The  opposite  was  true  for  the  experts,  relative  to  the  novices. 

Level  had  a  very  strong  influence  on  the  target  sentence  ratings,  as  shown  by  the 
large  parameter  estimates  for  this  effect.  Level  1  target  sentences  tended  to  be  rated  as 
most  important,  indicated  by  the  large  positive  estimate  for  the  rating  1  category.  The 
ratings  for  level  2  target  sentences  were  spread  over  the  categories  without  any  strong 
clustering  in  any  one  rating;  none  of  the  parameter  estimates  for  level  2  sentences  differed 
from  zero  by  more  than  two  standard  errors.  Level  3  target  sentences  tended  to  be  rated 
as  3  or  4  in  importance,  indicated  by  the  positive  parameters  for  these  ratings  categories. 
Level  did  not  interact  with  type  or  group  in  influencing  the  ratings  (i.e. ,  including  these 
effects  reduced  the  fit  of  the  model) 

Ratings  data :  Control  groups  The  multiway  frequency  analysis  of  the  control  group 
data  indicated  that  the  best-fitting  model  included  only  the  main  effects  of  group  and  level 
(^( 2  =  28.64.  df~ 40.  pc. 91).  The  inclusion  of  the  type  x  group  interaction  or  the  main  effect 
of  type  reduced  the  fit  of  the  model  to  the  data  set.  The  mean  ratings  predicted  by  the 
model  with  the  type  x  group  interaction  included  are  presented  in  Figure  1b  for  comparison 
with  the  corresponding  experimental  group  means.  As  Figure  1b  shows,  there  was  no 
difference  between  the  undergraduate  and  graduate  controls  in  the  influence  of  sentem.*.' 
type  on  the  mean  ratings  of  the  target  sentences  Additionally,  the  lack  of  a  main  effect  of 
sentence  type  indicates  that  the  form  in  which  the  target  sentences  were  presented  did  not 
influence  the  control  group  ratings  (i.e..  they  did  not  consider  definitions  to  be  important 
than  facts). 
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The  parameter  estimates  for  the  loglinear  model  including  the  main  effects  of  group 
and  level  are  shown  in  Table  3.  The  estimates  for  the  main  effect  of  group  indicate  that 
the  undergraduates  and  graduates  differed  primarily  in  their  use  of  ratings  1  and  4.  The 
positive  estimate  for  the  undergraduates  for  rating  category  1  indicates  that  they  rated  target 
sentences  1  more  often  than  the  graduate  controls  The  opposite  was  true  for  the  rating  4 
category,  with  the  graduates  using  this  rating  more  often  relative  to  the  undergraduates. 

This  indicates  that  undergraduates  rated  the  sentences  as  more  important  overall  than  the 
graduate  controls.  This  effect  is  also  apparent  in  Figure  1b. 

Insert  Table  3  about  here 


The  pattern  of  parameter  estimates  for  the  main  effect  of  level  is  similar  to  that 
obtained  with  the  experimental  groups.  Level  1  target  sentences  were  most  likely  to  receive 
a  rating  of  1,  indicated  by  the  large  positive  parameter  estimate  for  that  rating.  The  ratings 
given  to  Level  2  target  sentences  were  spread  over  the  categories,  with  the  strongest 
clustering  in  the  rating  1  category  (though  this  parameter  estimate  was  much  smaller  than 
the  rating  1  estimate  for  level  1  sentences).  Level  3  target  sentences  were  most  likely  to 
receive  a  rating  of  4,  with  3  as  the  next  most  frequent  rating  category  for  this  level. 

Ratings  data:  Summary.  The  results  of  the  multiway  frequency  analyses  indicate  that 
novices  base  their  judgements  of  the  importance  of  text  information  on  the  category 
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membership  of  the  content.  They  specifically  rate  the  same  attributive  information  as  more 
important  when  it  is  presented  as  a  definition  opposed  to  a  fnot  qc  is  shown  in  Ro'im 
la.  Expert  physicists  are  not  influenced  by  category  in  rating  importance,  presumably 
basing  their  importance  judgements  on  the  nature  of  the  sentence  content.  Similarly,  the 
physics-naive  control  subjects  are  not  influenced  by  signalled  category  membership,  so  that 
they  appear  to  behave  like  experts  in  rating  importance  This  can  be  seen  in  Figure  1b. 
This  is  most  likely  because  these  subjects  have  no  strong  expectations  about  the  relative 
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importance  of  definitions  and  facts  in  physics  texts,  and  thus  are  not  influenced  by  this  text 
feature.  The  lack  of  a  type  x  group  interaction  in  the  control  group  data  Indicate  that 
expert-novice  differences  in  the  perceived  importance  of  definitions  and  facts  are  not  due  to 
differences  in  educational  level 

Sentence  selection  data  Experimental  groups.  The  sentence  selection  data  were 
analyzed  using  a  logistic  regression.  The  regression  analysis  indicated  that  a  good  fit  to  the 
experimental  group  data  was  provided  by  a  hierarchical  model  including  the  group  x  type 
interaction  and  the  main  effect  of  level  (X 2  =  8.69.  df=  7,  p<  .28).  This  is  the  same  model 
found  to  provide  the  best  fit  for  the  ratings  data  from  the  experimental  groups.  The 
predicted  mean  proportion  of  target  sentences  selected  for  the  type  x  group  interaction  are 
shown  in  Figure  2a  plotted  on  a  logit  scale  This  interaction  is  very  similar  to  the  type  x 
group  interaction  obtained  with  the  ratings  data.  It  shows  that  the  novices  were  more  likely 
to  select  a  target  sentence  as  important  when  it  was  presented  in  the  form  of  a  definition 
than  a  fact,  while  the  experts  were  relatively  unaffected  by  sentence  form  in  their  selection 
of  the  important  sentences. 


Insert  Figure  2  about  here 


The  parameter  estimates  for  the  complete  logistic  regression  model  are  shown  in  Table 
4.  Unlike  the  estimates  for  the  multiway  frequency  analysis,  the  logistic  regression  estimates 
show  the  size  of  the  difference  between  the  means  for  the  two  variables  in  an  effect 
Therefore  only  one  parameter  extirpate  is  prneontoq  for  earh  main  effort  ft  >*/n  for  tho 
interaction)  and  the  ratios  indicate  the  size  of  the  difference  between  the  two  variables  in 
each  effect  and  interaction. 


Insert  Table  4  about  here 
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The  negative  estimate  for  the  main  effect  of  type  indicates  that  subjects  tended  to 
select  rrmre  target  sentences  when  they  were  presented  in  the  form  of  a  definition.  This  is 
consistent  with  the  main  effect  of  type  found  with  the  ratings  data  from  the  experimental 
groups.  The  estimate  for  the  main  effect  of  group  indicates  little  difference  between  the 

experts  and  novices  in  the  number  of  target  sentences  selected  as  important.  The  negative 
estimate  indicates  that  the  novices  selected  more  target  sentences  than  the  experts. 

However,  the  parameter  estimates  for  the  two  groups  differed  by  less  than  two  standard 

errors. 

The  type  x  group  interaction  estimates  indicate  that  novices  were  more  likely  to  select 
target  sentences  as  important  when  they  were  in  the  form  of  a  definition  rather  than  a  fact. 
Relative  to  novices,  the  experts  were  more  likely  to  select  the  sentences  when  they  were  in 
the  form  of  a  fact.  These  findings  are  shown  by  the  negative  estimate  for  definitions  and 
the  positive  estimate  for  facts. 

The  very  large  parameter  estimates  for  the  main  effect  of  level  indicates  that  this 
variable  had  a  strong  effect  on  which  target  sentences  were  selected  as  important.  The 
negative  estimate  shows  that  target  sentences  from  the  upper  levels  of  the  passage 

hierarchy  (level  1)  were  selected  more  often  than  the  target  sentences  from  the  lower  levels 
(level  3)  This  finding  is  also  consistent  with  the  strong  levels  effect  found  in  the  ratings 
data  As  with  the  ratings  data,  level  did  not  interact  with  sentence  type  or  group  in 
influencing  sentence  selection 

Sentence  selection  data  Control  groups  The  logistic  regression  for  the  control  group 
data  indicated  that  the  best-fitting  model  was  a  hierarchical  model  including  the  main  •  H>  ■  i 
of  group  and  the  type  x  level  interaction  t.V  1  78  dt=  5  p<  88)  The  inclusion  of  the 
type  x  group  interaction  reduced  the  fit  of  the  model  to  the  data  Thus  there  was  no 
evidence  that  the  importance  judgements  of  the  undergraduate  and  graduate  control  groups 
differed  in  the  degree  to  which  they  were  influenced  by  sentence  type  This  can  be  seen 
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in  Figure  2b.  which  shows  the  predicted  mean  proportion  of  definitions  and  facts  selected 
by  the  two  control  groups  plotted  on  a  logit  scale.  These  means  are  based  on  the  model 
with  the  type  x  group  interaction  included. 

The  parameter  estimates  for  the  regression  model  including  the  effects  of  group  and 
the  type  x  level  interaction  are  presented  in  Table  5.  Unlike  the  experimental  groups,  level 
did  not  have  a  linear  effect  on  the  number  of  target  sentences  selected  by  the  control 
groups.  It  was  therefore  entered  into  the  analysis  as  a  categorical  (as  opposed  to  a  linear) 
variable,  and  separate  parameter  estimates  were  obtained  for  each  level.  The  parameter 
estimates  presented  for  the  main  effect  of  level  represent  the  size  of  the  difference  between 
levels  1  and  2,  and  levels  2  and  3  For  the  type  x  level  interaction,  the  parameter 
estimates  indicate  the  size  of  the  difference  between  definitions  and  facts  at  each  level. 


Insert  Table  5  about  here 

The  negative  parameter  estimate  for  the  main  effect  of  type  shows  that  the  control 
groups  were  more  likely  to  select  a  target  sentence  when  it  was  in  the  form  of  a  definition 
than  a  fact.  However,  the  type  x  level  interaction  indicates  that  the  effect  of  type  varied 
with  level  The  negative  parameter  estimate  for  level  1  indicates  that  facts  were  selected 
more  often  than  definitions  at  the  top  level,  while  the  positive  parameters  for  levels  2  and  3 
show  that  definitions  were  more  likely  to  be  selected  than  facts  at  the  lower  levels  The 
predicted  cell  means  for  this  interaction  are  presented  in  Figure  3  Figure  3  shows  that  the 
main  effect  nf  sentence  tvpe  is  due  primarily  to  a  >'orv  irtro°  contenco  form  effort  at  loyoi  n 


Insert  Figure  3  about  here 


This  finding  suggests  that  sublets  without  physics  training  may  tend  to  |udge  details  (low- 
level  information)  in  physics  texts  as  being  more  important  when  they  are  presented  as 
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definitions  than  facts.  However,  this  sentence  type  difference  is  based  on  very  few  data 
points,  as  most  level  3  sentences  were  not  selected  as  important.  The  mean  number  of 
sentences  from  level  3  selected  by  the  control  groups  were  2.3  for  the  undergraduates  and 
1.6  for  the  graduates.  Additionally,  this  type  x  level  interaction  was  not  found  in  the  ratings 
data  for  the  control  groups.  Therefore,  it  is  possible  that  this  particular  effect  Is  not 
replicable. 

The  negative  parameter  estimate  for  the  main  effect  of  group  indicates  that 
undergraduates  selected  more  target  sentences  as  important  than  the  graduate  control 
group.  This  is  consistent  with  the  finding  in  the  ratings  data  that  undergraduates  tended  to 
rate  the  target  sentences  higher  in  importance  relative  to  the  graduate  controls. 

The  positive  parameter  estimates  for  the  main  effect  of  level  indicate  that  averaged 
across  sentence  type,  the  number  of  sentences  selected  as  important  decreased  with  level 
The  size  of  the  estimates  show  that  the  drop  in  the  number  of  target  sentences  selected  as 

important  was  much  greater  between  levels  2  and  3  than  between  levels  1  and  2.  This  can 

also  be  seen  in  Figure  3. 

Sentence  selection  data  Summary.  The  results  of  the  sentence  selection  task 
analyses  are  consistent  with  the  findings  from  the  ratings  data.  Sentence  category 
influenced  the  importance  judgements  of  novices,  with  novices  selecting  more  target 
sentences  when  they  were  presented  in  the  form  of  a  definition  (see  Figure  2a).  Sentence 
category  had  very  little  effect  on  the  sentences  selected  by  experts.  There  again  was  no 
type  x  group  interaction  in  the  control  group  data  as  shown  in  Figure  2b.  indicating  that 
expert-novice  differences  are  not  due  lo  differences  in  educational  level  Sentence  cu-'got> 

did  have  some  influence  on  the  judged  importance  of  sentences  for  the  control  groups  in 

that  they  tended  to  select  more  of  the  definition  versions  than  fact  version  nom  level  3 
Thus  it  is  possible  that  there  is  a  bias  towards  considering  low-level  definitions  as  important 
in  subjects  who  do  not  have  scientific  backgrounds 
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Experiment  2 

Experiment  1  demonstrated  that  novice  importance  rules  are  based  on  category 
membership,  and  not  content  differences  between  categories.  Novices  specifically  develop  a 
rule  that  definitions  are  particularly  important;  as  a  result,  they  judge  the  same  attributive 
information  as  more  important  when  it  is  presented  in  the  form  of  a  definition,  The  purpose 
of  the  second  experiment  was  to  replicate  and  extend  these  findings  by  examining  the 
influence  of  sentence  form  on  the  judged  importance  of  quantitative  relations. 

Quantitative  relations  are  particularly  important  for  understanding  physics  in  that  they 
are  central  to  solving  problems.  The  ability  to  recognize  problem-relevant  quantitative 
information  is  essential  for  efficient  problem-solving  However,  quantitative  relations  vary  in 
the  manner  in  which  they  are  presented  in  texts.  They  can  be  presented  as  equations,  a 
form  which  explicitly  signals  the  quantitative  nature  of  the  content,  or  they  can  be  written 
out  as  verbal  formulae  For  example,  the  quantitative  relation  between  a.  b.  and  c  can  be 
expressed  as  "a  =  b/c"  (an  equation;  or  written  out  as  "a  is  equal  to  b  divided  by  c”  (a 
verbal  statement) 

Efficient  and  accurate  problem  solving  requires  that  the  student  identify  those 
quantitative  relations  relevant  to  the  problem,  regardless  of  the  form  in  which  they  are 
presented  However,  the  results  of  research  with  uncontrolled  passages  (Dee-Lucas  & 

Larkin,  1986)  suggest  that  novices  may  consider  equations,  like  definitions,  to  be  a 
particularly  important  category  of  information  That  is.  novices  may  have  a  rule  that 
enuations  am  particularly  important  (similar  to  their  rule  for  Hofinitionsl  which  causes  them  to 
consider  content  presented  in  equational  form  as  "automatically"  important  If  so  this 

importance  rule  could  have  important  implications  for  novice  problem  solving  performance 
Novices  could  be  biased  in  their  assessment  of  the  relevance  of  quantitative  relations  for 
problem  solving  depending  on  the  form  in  which  that  information  was  presented  They 
would  consider  equations  to  be  more  important  and  therefore  more  likely  to  be  problem- 
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relevant,  than  quantitative  relations  presented  in  verbal  form. 

Experiment  2  determined  whether  novices  develop  a  category-based  rule  that  equations 
are  particularly  important  in  physics  texts.  This  was  done  by  comparing  expert  and  novice 
judgements  of  the  importance  of  quantitative  relations  when  they  were  expressed  as 
equations  or  written  out  in  verbal  form.  The  purpose  of  the  experiment  was  twofold.  First, 
it  provided  a  replication  of  Experiment  1  in  support  of  the  general  finding  that  novices  are 
influenced  by  sentence  form  in  judging  importance.  Second,  it  extended  this  finding  to  a 
type  of  content  which  is  central  for  problem  solving  as  well  as  text  learning  in  physics. 

In  the  current  study,  experts  and  novices  read  passages  containing  target  sentences 
which  presented  quantitative  relations  as  either  equations  or  verbal  statements.  Subjects 
rated  the  importance  of  each  sentence  on  a  5  point  scale,  and  selected  the  10  most 
important  sentences  from  each  passage  The  judged  importance  of  the  quantitative  relations 
when  they  were  presented  as  equations  and  verbal  statements  was  compared  to  see  if 
novices  were  influenced  in  their  judgements  by  sentence  form. 

Method 

Stimulus  Materials  The  two  passages  were  about  fluid  statics,  and  work  and  energy. 
Each  was  approximately  50  sentences  long  and  each  contained  9  target  sentences. 
Examples  of  the  equation  and  verbal  forms  of  the  target  sentences  are  shown  in  Table  1b. 
Symbols  for  quantities  were  used  in  both  versions  but  the  relation  itself  was  expressed 
symbolically  in  the  equation  form  and  written  out  in  the  verbal  form  Thus  the  informational 
content  was  identical,  and  only  the  form  in  whioh  tho  rnintinn  was  ronvpvod  li  p  svmdniir 
or  verbal)  differed  There  were  two  versions  of  each  passage  In  one  version,  the  odd- 
numbered  target  sentences  were  equations  and  the  even-numbered  were  verbal  statements, 
in  the  other  version  this  was  reversed 

As  m  Experiment  1.  each  of  the  target  sentences  was  classified  according  to  its  level 
in  the  passage  hierarchy  There  were  6  sentences  at  each  of  the  first  3  levels  in  the 
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passage  hierarchies.  Level  was  included  as  a  variable  in  the  data  analyses  to  determine  if 
perceived  importance  was  influenced  by  level,  and  if  level  interacted  with  sentence  form  (i.e.. 
equation  or  verbal  statement). 

Subjects.  The  novices  were  18  undergraduates  who  had  completed  2  semesters  of 
college  physics.  The  18  experts  had  completed  at  least  one  year  of  graduate  study  in 
physics. 

Two  control  groups  were  again  run  to  determine  if  expert-novice  differences  were  due 
to  differences  in  educational  level  rather  than  differences  in  physics  knowledge.  The 
undergraduate  control  group  consisted  of  18  undergraduates:  the  graduate  control  group 
consisted  of  18  graduate  students  who  had  completed  at  least  one  year  of  graduate  study 
in  the  humanities  or  social  sciences  None  of  the  control  group  subjects  had  taken  any 
college-level  physics 

Procedure.  The  procedure  and  instructions  were  the  same  as  for  Experiment  1. 

Results 

The  data  were  analyzed  in  the  same  manner  as  the  data  from  Experiment  1, 

Ratings  data  Experimental  groups.  The  multiway  frequency  analysis  of  the  ratings 
data  for  the  experimental  groups  indicated  that  the  best-fitting  hierarchical  model  included 
the  type  x  group  interaction  and  the  main  effect  of  level  ( A 2  =  17.81.  df=  30.  pc. 96).  Figure 
4a  shows  the  mean  ratings  predicted  by  the  model  for  the  type  x  group  interaction.  As  in 
Experiment  1.  novices  were  influenced  by  sentence  form  while  experts  were  not  Novices 
considered  the  same  information  to  h°  more  important  whop  it  was  ovproecort  as  an 
equation  as  opposed  to  a  verbal  statement  The  predicted  means  for  the  novices  were  1.44 
for  the  equation  versions  and  i  92  for  the  verbal  versions  For  experts,  these  means  were 
1  61  and  1  68  respectively. 


Insert  Figure  4  about  here 
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The  parameter  estimates  for  the  complete  model  are  shown  in  Table  6  The 
estimates  for  the  main  effect  of  group  show  little  difference  between  experts  and  novices  in 
their  use  of  the  different  ratings  Novices  had  a  slightly  greater  tendency  to  use  the  rating 
2  category  relative  to  experts,  while  experts  tended  to  use  the  rating  4  category  more  often, 
but  the  magnitude  of  these  effects  are  not  large  (i.e. .  do  not  differ  from  zero  by  more  than 
2  standard  errors). 


Insert  Table  6  about  here 


The  estimates  for  the  main  effect  of  type  show  a  large  difference  in  the  overall  ratings 
of  equations  and  verbal  formulae.  Equations  were  much  more  likely  than  verbal  statements 
to  receive  the  highest  importance  rating  (rating  1).  and  somewhat  less  likely  to  receive  a 
rating  of  2  and  4.  This  indicates  that  equations  were  rated  higher  in  importance  overall 
than  verbal  formulae. 

The  type  x  group  interaction  estimates  indicate  that  novices  were  much  more  likely  to 
give  a  rating  of  1  to  equations  than  verbal  formulae,  and  somewhat  more  likely  to  give  low 
rating  of  4  to  verbal  formulae  than  equations.  The  opposite  was  true  for  experts  relative  to 
novices. 

The  level  parameter  estimates  again  show  that  level  has  a  strong  effect  on  judged 
importance.  Subjects  tended  to  give  level  1  target  sentences  a  rating  of  1.  level  2  target 
sentences  a  ratinci  of  2.  and  level  3  tamet  apntonres  a  ration  of  «t  l  owol  did  not  imoraot 
with  type  or  group 

Ratings  data  Control  groups  The  multiway  frequency  analysis  of  the  ratings  data  for 
the  control  groups  indicated  that  the  most  appropriate  model  included  the  main  effects  of 
type,  level,  and  group  (T2  =  23.40.  dt  =  35.  p>  93)  The  inclusion  of  the  type  x  group 
interaction  in  the  model  reduced  its  fit  to  the  data  set.  Thus  the  effect  of  sentence  form 
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on  rated  importance  did  not  vary  with  level  of  expertise.  This  can  be  seen  in  Figure  4b, 
which  shows  the  mean  ratings  predicted  by  the  model  with  the  type  x  group  interaction 
included.  The  difference  in  the  ratings  of  the  equation  and  verbal  sentence  forms  did  not 
vary  with  the  educational  level  of  the  control  groups. 

The  parameter  estimates  for  the  model  including  the  main  effects  of  group,  type,  and 
level  are  shown  in  Table  7.  The  estimates  for  the  main  effect  of  sentence  type  indicates 
that  sentence  form  did  influence  the  importance  ratings  of  the  control  groups.  Equations 

were  more  likely  to  be  rated  1  and  verbal  formulae  rated  2  This  can  also  be  seen  In 

Figure  4b.  which  shows  that  both  groups  judged  information  as  more  important  when  it  was 
presented  in  the  equational  form  as  opposed  to  the  verbal  form.  This  was  not  the  case  in 
Experiment  1.  in  that  the  judged  importance  of  the  target  content  did  not  vary  when  it  was 
presented  in  the  form  of  a  definition  or  fact  This  indicates  that  there  are  general  rules 

formulated  by  the  population  at  large  regarding  the  relative  importance  of  different  types  of 

content  in  certain  types  of  texts.  In  this  case  the  physics-naive  as  well  as  novice  subjects 

considered  equations  to  be  particularly  important.  However,  as  shown  in  Figure  4.  the 

difference  in  the  judged  importance  of  equational  and  verbal  statements  was  much  greater 
with  novices  than  controls,  indicating  that  this  general  rule  developed  by  physics-naive 

subjects  is  accentuated  by  limited  physics  training,  and  decreases  with  extensive  physics 

study  (as  seen  in  the  expert  data). 


Insert  Table  7  about  here 


The  estimates  for  the  main  effect  of  group  show  that  there  were  large  differences 
between  the  two  groups  in  their  use  of  ratings  1.  3  and  4.  The  undergraduate  controls 
tended  to  rate  the  sentences  either  very  high  in  the  importance  (1)  or  very  low  in 


importance  (4).  while  the  graduate  controls  were  more  likely  to  use  the  intermediate  rating 
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The  estimates  for  level  show  the  usual  level  effects.  Subjects  rated  the  top-level 

target  sentences  high  in  importance  (rating  t).  and  low  level  sentences  low  in  importance 

(rating  5).  The  ratings  for  level  2  sentences  did  not  tend  to  cluster  in  any  one  rating 

category. 

Ratings  data:  Summary.  The  ratings  data  comparing  the  judged  importance  of 

equations  and  verbal  statements  confirm  the  findings  from  Experiment  1.  In  both 
experiments,  novices  were  sensitive  to  sentence  form  while  experts  were  not.  Specifically, 
when  sentence  information  was  signalled  as  belonging  to  a  certain  information  category  (i.e. . 
definition  or  equation),  novices  rated  that  content  higher  in  importance  than  when  category 
membership  was  changed  to  that  of  another  category,  or  made  less  prominent.  This  "form 
effect"  reflects  novice  rules  regarding  the  importance  of  different  types  of  content  in  physics 


Unlike  Experiment  1.  the  control  group  data  showed  that  physics-naive  subjects  were 
also  influenced  by  sentence  form  in  the  presentation  of  quantitative  information.  This 
suggests  that  there  is  a  general  consensus  that  equations  are  particularly  important  in 
science  texts;  such  a  consensus  apparently  does  not  exist  for  definitions.  Note,  however, 
that  this  initial  preconception  about  the  importance  of  equations  is  strengthened  by  a  limited 
amount  of  physics  training  (see  Figure  5a  and  b).  The  difference  in  the  rated  importance 
of  the  equation  and  verbal  forms  of  the  target  sentences  is  much  greater  for  novices  than 
physics-naive  subjects.  The  expert  data  additionally  shows  that  after  extensive  physics 
training,  subjects  recognize  the  importance  of  quantitative  information  regardless  of  whether  it 
is  presented  as  an  equation  or  verbal  formula,  and  thus  are  not  influenced  by  information 
form.  The  lack  of  a  type  x  group  interaction  in  the  control  group  data  again  indicates  that 
expert-novice  differences  are  not  due  to  differences  in  educational  level. 

Sentence  selection  data.  Experimental  groups  The  logistic  regression  for  the  sentence 
selection  data  from  the  experimental  groups  indicated  that  the  best-fitting  model  was  a 
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hierarchical  model  including  the  the  type  x  group  interaction  and  the  main  effect  of  level 
(X2  =  10.32,  df=  7,  p< .17).  This  is  the  same  model  that  accounted  for  the  experimental 
group  data  from  Experiment  1  and  the  experimental  ratings  data  in  the  current  study.  The 
predicted  mean  proportion  of  target  sentences  selected  for  each  group  and  type  are  shown 
in  Figure  5a  plotted  on  a  logit  scale  This  interaction  shows  that  novices  are  much  more 
likely  to  select  a  target  sentence  as  important  when  it  is  presented  in  the  form  of  an 
equation  as  opposed  to  a  verbal  formula  while  experts'  sentence  selection  is  not  influenced 
by  sentence  category.  This  is  consistent  with  the  results  of  Experiment  1  and  the  ratings 
data  indicating  that  novices  are  influenced  by  sentence  form,  while  experts  are  not 

Insert  Figure  5  about  here 


The  parameter  estimates  for  the  logistic  regression  model  are  shown  in  Table  8.  As 
in  Experiment  1,  these  indicate  the  size  of  the  difference  between  the  means  for  an  effect. 
The  negative  parameters  for  the  main  effects  of  type  and  group  indicate  that  summed 
across  groups,  equations  were  selected  more  often  than  verbal  formulae,  and  that  novices 
selected  more  of  the  target  sentences  overall  than  the  experts. 

The  type  x  group  interaction  estimates  show  that  novices  were  more  likely  to  select 
target  sentences  as  important  when  they  were  presented  as  equations  as  opposed  to  verbal 
formulae.  Relative  to  novices,  experts  were  more  likely  to  pick  sentences  in  the  verbal  form, 
as  indicated  by  the  postive  parameter  for  verbal  formulae  and  the  negative  parameter  for 
equations. 

The  large  negative  parameter  estimate  for  level  replicates  previous  results  indicating 
that  subjects  are  much  more  likely  to  select  high-level  sentences  than  low-level  sentences  as 
important. 

Sentence  selection  data  Control  groups  The  most  appropriate  model  for  the  logistic 
regression  for  the  control  group  data  was  a  hierarchical  model  including  the  main  effect  of 
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group  and  the  type  x  level  interaction  {X 2  =  7.02,  df=  5,  pc. 22).  The  inclusion  of  the  type  x 
group  interaction  resulted  in  a  poorer  fit  of  the  model  to  the  data.  Thus  the  two  control 
groups  did  not  differ  from  each  other  in  their  evaluation  of  the  importance  of  the  equations 
and  their  verbal  equivalents  This  can  be  seen  in  Figure  5b,  which  shows  the  predicted 

mean  proportion  of  target  sentences  selected  for  the  type  x  group  interaction,  based  on  the 

model  with  the  interaction  term  included.  Figure  5b  shows  that  both  undergraduates  and 
graduate  students  picked  equations  more  often  than  verbal  formulae.  This  is  consistent  with 
the  results  of  the  ratings  data  (shown  in  Figure  4b). 

The  parameter  estimates  for  the  regression  model  including  the  main  effects  of  group 
and  type  x  level  interaction  are  shown  in  Table  9.  As  in  Experiment  1,  level  did  not  have  a 
linear  effect  on  the  number  of  target  sentences  selected  by  the  control  groups,  and 

therefore  was  entered  as  a  categorical  variable  in  the  analysis.  The  parameter  estimates 
show  the  size  of  the  difference  between  levels.  The  type  x  level  parameter  estimates 

indicate  the  size  of  the  difference  between  equations  and  verbal  formulae  for  each  level. 


Insert  Table  9  about  here 


The  parameter  estimate  for  the  main  effect  of  type  shows  that  overall  the  controls 
selected  more  equations  than  verbal  formulae  as  important.  However,  the  type  x  level 
interaction  indicates  that  the  effect  of  sentence  form  varied  with  level,  with  the  largest 
difference  between  sentence  types  occuring  at  level  2.  Figure  6  shows  the  predicted  mean 

proportion  Of  sontonpes  spfnptod  for  fhi>  information  ThO  prrforonro  fnr  nqmfinnq  omn  ho 
seen  for  levels  1  and  2.  with  little  difference  in  sentence  type  selection  at  level  3  Thus 
the  general  tendency  for  physics-naive  subjects  to  consider  equations  as  important  may 
occur  primarily  for  higher-level  text  information  rather  than  text  details.  However,  this 
interaction  was  not  found  in  the  ratings  data  and  may  in  part  reflect  constraints  imposed  by 
the  limited  choice  (i.e. .  pick  10)  dependent  measure.  While  many  sentences  were  chosen 


•  Novice  Rules 


25 


from  levels  1  and  2,  relatively  few  sentences  of  any  type  were  selected  from  level  3.  Thus 
the  lack  of  a  difference  at  level  3  may  be  partly  due  to  a  floor  effect. 


Insert  Figure  6  about  here 


The  negative  parameter  for  the  main  effect  of  group  shows  that  the  undergraduates 
tended  to  select  more  target  sentences  as  important  than  the  graduate  students.  This  is 
consistent  with  the  ratings  data  indicating  that  the  undergraduates  rated  the  target  sentences 
more  important  overall  than  the  graduate  controls. 

The  two  positive  parameter  estimates  for  level  indicates  that  the  controls  selected  more 
sentences  from  the  higher  levels  than  lower  levels  of  the  texts.  The  size  of  the  coefficients 
show  that  summed  across  sentence  types,  the  decrease  between  levels  1  and  2  in  the 
number  selected  was  roughly  equal  to  the  decrease  between  levels  2  and  3.  This  can  also 
be  seen  in  Figure  6. 

Sentence  selection  data  Summary  The  data  from  the  sentence  selection  task  again 
supports  the  findings  from  Experiment  1  that  novices  are  sensitive  to  variations  in  sentence 
form  when  these  variations  signal  specific  category  membership.  Novices  judged  the  same 
quantitative  relations  as  more  important  when  they  were  presented  as  equations  as  opposed 
to  being  written  out  in  verbal  form  Experts  were  not  influenced  in  their  importance 

judgements  by  sentence  form. 

As  in  the  ratings  data,  the  control  subjects  also  considered  the  equational  form  of  the 
tarnet  sentences  to  he  more  imnor  3  effort  was  confined  to  tho  firct  two  lowoic  of 

the  passage  hierarchy  in  the  sente  selection  data  This  again  indicates  that  physics- 

naive  people  also  consider  equations  i  be  particularly  important  in  science  texts,  and  judge 
quantitative  relations  as  more  importar  when  they  ate  presented  in  this  form.  However,  as 
with  the  ratings  data,  the  influence  of  sentence  form  was  much  greater  on  novices  than 

physics-naive  subjects,  indicating  that  this  preconception  is  strengthened  by  limited  physics 
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training.  In  Experiment  1.  there  was  not  a  general  tendency  for  physics-naive  people  to 
view  definitions  as  more  important  than  facts  (with  the  possible  exception  of  a  small  effect 
for  low  level  content).  These  findings  suggest  that  there  may  be  domain-specific  category 
importance  rules  in  the  population  at  large,  but  these  rules  are  weaker  and  possibly  fewer  in 
number  than  those  generated  by  novices  as  a  result  of  limited  domain-related  training 

Discussion 

Although  much  research  has  examined  the  knowledge  representations  of  experts, 
relatively  little  is  known  about  the  knowledge  structures  of  novices  However,  how  novices 
represent  unfamiliar  content  domains  and  the  nature  of  the  changes  occurring  in  these 
representations  has  important  educational  implications  for  facilitating  learning  from  text. 
Characterizations  of  the  types  of  knowledge  structures  used  by  novices  to  govern  text 
processing  indicate  how  text  should  be  structured  both  globally  and  locally,  to  maximize  the 
probability  that  novices  will  learn  the  important  text  content 

This  study  examined  one  aspect  of  novice  knowledge  representations  -  rules  for 
assessing  importance  in  unfamiliar  scientific  domains  Previous  research  has  shown  that 
novices  develop  rules  specifying  what  types  of  information  are  important  in  science  texts 
The  current  research  investigated  the  nature  of  these  novice  importance  rules.  It  provided  a 
strong  test  of  the  hypothesis  that  novice  importance  rules  are  based  on  information-type 
categories,  in  that  all  variables  other  than  information  category  were  held  constant.  This 
was  done  by  using  minor  wording  changes  to  alter  the  category  membership  of  sentences, 
nnH  lonkinn  Of  how  thir  rhnnnnH  nornn-'or)  'Olp^rtgnCC 

The  results  indicate  that  novices  are  influenced  by  category  membership  in  judging 
importance  Their  judgements  of  the  importance  of  content  varies  according  to  the  form  in 
which  the  content  is  presented  They  consider  the  same  attributive  information  to  be  more 
important  when  it  is  presented  as  a  definition  as  opposed  to  a  fact,  and  the  same 
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quantitative  relations  to  be  more  important  when  they  are  presented  as  equations  as 
opposed  to  verbal  formulae  This  "form  effect"  is  not  seen  in  experts,  and  Is  either  absent 
or  attenuated  in  physics-naive  subjects  Thus  novices  are  sensitive  to  variations  in  sentence 
form  when  these  variations  indicate  membership  in  particular  information  categories. 

These  findings  indicate  that  novice  importance  rules  identified  in  earlier  research  using 
uncontrolled  materials  are  based  on  category  membership,  and  not  content  differences 
between  categories  Thus,  for  example,  novices  judge  definitions  as  more  important  than 
facts  simply  because  they  are  definitions  However,  similar  differences  observed  in  experts 
with  uncontrolled  materials  do  reflect  content  differences  --  when  the  content  of  different 
categories  is  held  constant,  experts  judge  the  same  information  presented  in  different  forms 
to  be  equivalent  in  terms  of  its  importance  This  indicates  that  experts  judge  certain 
categories  as  more  important  than  others  because  these  categories  typically  contain  the  type 
of  content  that  is  important  for  understanding  physics  For  example,  experts  most  likely 
consider  definitions  to  be  particularly  important  because  it  is  necessary  to  fully  understand 
physics  terminology  in  order  to  understand  the  principles  discussed  in  a  text.  Thus  novices 
judge  according  to  the  information  category  itself  (assuming  that  the  content  is  important 
because  of  its  category  membership)  but  experts  judge  according  to  the  importance  of  the 
specific  content,  which  is  correlated  with  certain  information  categories  The  similarities 
between  novices  and  experts  in  their  importance  judgements  with  uncontrolled  texts  are 
therefore  based  on  qualitatively  different  analyses  of  the  text  content. 

These  findings  suggest  that  people  just  beginning  to  learn  about  a  content  domain 
develop  rules  specifying  what  categories  of  information  are  important  in  tint  domain  1 1" 
current  research  shows  that  novices  specifically  consider  definitions  and  equations  to  be 
particularly  important  in  physics  texts  These  rules  are  not  inappropriate  -  experts  also 
consider  definitions  and  equations  to  be  important  in  naturalistic  physics  texts,  indicating  that 
definitions  and  equations  typically  contain  information  that  is  particularly  important  for 
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understanding  physics  (Dee-Lucas  &  Larkin.  1986)  However,  the  results  of  the  current  study 
indicate  that  novices  judge  importance  on  the  basis  of  sentence  type  category  without 
regard  for  sentence  content  They  judged  the  same  content  as  more  important  when  it  was 
presented  as  a  definition  or  equation  indicating  that  they  consider  these  categories  of 
information  to  be  important  regardless  of  their  content.  This  suggests  that  information-type 
rules  developed  by  novices  are  applied  too  rigidly,  in  that  novices  are  not  distinguishing 
between  important  and  unimportant  information  within  type  categories.  In  this  sense,  novice 
rules  are  "overgeneralized."  This  overgeneralization  results  in  novices  systematically  mis- 
identifying  the  important  text  content 

The  results  of  the  current  research  suggest  how  a  content  schema  for  a  new 
knowledge  domain  might  evolve  in  novice  learners.  A  content  schema  indicates  how  the 
knowledge  for  a  particular  domain  is  typically  structured,  including  what  is  important  in  the 
content  area  The  current  study  suggests  that  an  early  stage  in  the  development  of  a 
content  schema  for  scientific  subject  matter  may  be  the  specification  of  rules  indicating  what 
categories  of  information  are  important  in  that  content  domain.  These  rules  could  form  the 
foundation  for  developing  an  expert  content  schema  by  providing  a  relatively  undifferentiated 
base  from  which  a  more  refined,  hierarchical  rule  structure  could  be  developed.  In 
developing  an  expert  content  schema,  novices  would  move  from  a  classification  system 
based  on  information  categories  to  a  more  refined  knowledge  structure  based  on  a  deeper 
analysis  of  the  nature  of  the  category  content  Starling  with  a  novice  schema  composed  of 
a  few  general  category  rules,  novices  could  begin  to  differentiate  the  important  and  less 
important  information  within  categories  as  they  learn  about  the  importance  of  specific 
content.  In  this  way  they  could  gradually  move  to  a  classification  system  based  on 
commonalities  among  important  content  from  various  information  categories  For  example, 
novices  could  begin  to  view  relational  information  as  potentially  important,  regardless  of  its 
superficial  category  membership  (i  e  .  whether  it  is  specified  as  a  fact,  equation,  definition. 
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etc.). 

This  type  of  schema  shift  has  been  found  by  Chi,  Feltovich,  and  Glaser  (1981)  and 
Schoenfeld  and  Herrmann  (1982)  in  research  on  problem  solving  in  physics  and  math  They 
found  that  novices  classify  physics  and  math  problems  on  the  basis  of  superficial 
characteristics  of  the  problems,  such  as  the  objects  and  terms  mentioned  in  the  problem, 
and  experts  classify  according  to  the  underlying  principles  used  in  solving  the  problems 
Additionally.  Chi.  et  al..  found  that  subjects  of  intermediate  expertise  use  both  dimensions  in 
their  classification  systems,  suggesting  that  there  may  be  a  gradual  evolution  from  the 
problem-features  defined  as  relevant  by  novices  to  those  used  by  experts  Chi.  et  al.. 

characterize  this  shift  as  moving  from  "surface  structure"  features  to  a  "deep  structure" 
characterization  in  analyzing  physics  problems  The  results  of  the  current  study  suggest  that 
there  is  a  similar  shift  in  the  text  features  viewed  as  relevant  for  assessing  importance  by 
novices  and  experts.  Novices  consider  the  surface-level  feature  of  category  membership  as 
pertinent  to  judging  importance:  experts  rely  on  a  deeper  analysis  of  the  nature  of  the  text 
content. 

In  some  cases,  novice  importance  rules  may  be  extensions  of  general  preconceptions 
developed  before  formal  domain-related  training  In  the  current  study,  the  physics-naive 
control  subjects  as  well  as  the  novices  thought  that  equations  were  particularly  important, 

even  though  the  control  subjects  had  had  very  little  physics  experience.  This  finding  for  the 
control  groups  most  likely  reflects  a  general  preconception  about  science  texts  derived  from 
experience  with  a  broad  range  of  texts  in  this  class.  This  suggests  that  the  tendency  to 
develop  category-based  importance  rules  may  be  a  general  phenomenon,  probably  occurring 
in  most  content  domains  having  well-defined  easily-idenlifiable  categories  of  information 
Although  the  development  of  these  rules  does  not  appear  to  depend  on  formal  instruction 
these  initial  general  rules  can  be  strengthened  by  limited  formal  experience  within  a 

particular  content  area  In  the  current  research  both  control  subjects  and  novices  judged 
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information  as  more  important  when  it  was  presented  as  an  equation,  but  the  difference  in 
the  |udged  importance  of  equation  and  non-equation  forms  was  much  greater  for  the  novices 
than  the  controls  Thus  to  the  extent  that  these  early  general  preconceptions  are  accurate 
ti  e  do  reflect  to  a  certain  extent  what  is  important  in  the  domain),  they  appear  to  be 
accentuated  by  limited  formal  training  in  the  field.  It  is  also  likely  that  general 
preconceptions  which  are  inaccurate  would  be  eliminated  or  replaced  by  such  training. 

Although  the  control  subjects  appear  somewhat  sensitive  to  sentence  category 
membership,  this  sensitivity  is  not  as  consistent  nor  the  effect  as  large  as  it  is  for  novices. 
Thus  the  control  subjects  appear  very  similar  to  experts  This  is  most  likely  because  they 
have  few  expectations  about  what  types  of  information  are  important  in  physics  texts,  and 
those  that  they  do  have  are  not  very  strong  <i  e  .  not  as  strong  as  for  novice  subjects). 
Thus  the  control  subjects  are  not  as  strongly  affected  by  sentence  category  because  they 
lack  a  physics-relevant  content  schema  the  experts  are  not  influenced  by  sentence  category 
because  they  have  developed  a  much  more  refined  schema  which  indicates  the  importance 
of  specific  information  within  the  domain  of  physics  The  experts'  content  schema  includes 
a  finer-grained  analysis  of  the  importance  of  physics  knowledge  than  that  captured  by 
sentence-type  categories 

This  suggests  an  inverted  U-shaped  relationsh.p  among  naive,  novice,  and  expert 


physicists  in  their  sensitivity  to  sentence  category  membership  Both  naive  and  expert 
sublets  are  not  influenced  by  sentence  category  to  any  large  extent,  they  judge  the  same 
content  equal  or  similar  m  importance  regardless  of  its  form  Novices,  on  the  other  hand, 
are  distracted"  by  form  in  then  importance  judgements  in  that  they  greatly  aHei  tlicii 
judgements  according  to  the  specified  category  of  the  content  This  is  because  their 
knowledge  of  physics  has  caused  them  to  develop  specific  expectations  that  certain 
information  categories  are  more  important  than  others  (expectations  that  are  weak  or  lacking 
in  naive  subjects)  but  they  do  not  have  sufficient  knowledge  to  judge  the  importance  of 
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specific  content  within  these  categories  (as  experts  can),  and  thus  rely  on  these  general 
sentence  category  rules 

A  similar  U-shaped  relation  has  been  found  among  beginning  intermediate,  and  expert 
radiologists  in  diagnosing  x-ray  pictures  Lesgold.  Feltovich.  Glaser.  8  Wang  (1981;  see  also 
Lesgold.  1984)  found  that  beginning  residents  and  experts  were  better  than  intermediate-level 
radiologists  in  the  diagnostic  reading  of  certain  classes  of  x-ray  films.  This  is  because 
accurate  diagnosis  involves  an  interaction  between  the  physical  features  found  on  the  x-ray 
and  the  radiologist  s  knowledge  of  the  relevant  contextual  features  which  constrain  the 
possible  alternative  diagnoses  Beginning-level  radiologists  base  their  diagnoses  on  the 
physical  features  of  the  x-ray  and  are  accurate  when  there  happens  to  be  a  match  between 
their  interpretation  of  those  features  and  the  actual  pathology;  experts  use  their  schematic 
knowledge  to  interpret  the  physical  features  in  the  context  of  other  relevant  information  (such 
as  the  patient  s  medical  condition),  and  thus  are  systematically  accurate  in  their  diagnoses. 
Intermediate-level  radiologists  possess  some  schematic  knowledge,  but  this  knowledge  is  not 
refined  or  flexible  enough  to  provide  accurate  diagnoses  -  it  "distracts"  them  from  the 
direct  physical  features  used  by  beginners,  and  is  not  elaborate  enough  to  allow  them  to 
pinpoint  the  appropriate  alternative  in  the  same  manner  as  experts 

Thus  both  with  physicists  and  radiologists,  there  is  a  stage  in  the  development  of 
expertise  in  which  novices  possess  a  primitive  schema  incorporating  very  general  information 
about  the  domain  (i  e  what  categories  of  information  are  typically  important  for 
understanding  physics:  what  contextual  features  are  relevant  for  interpreting  an  x-ray).  These 
S'  lu'mns  are  not  sufficiently  refined  to  reliably  allow  accurate  performance,  and  actually 
impair  novice  performance  (relative  to  that  of  naive  subjects!  on  certain  tasks  Similar 
developmental  trends  have  been  found  in  the  performance  of  children  on  a  variety  of 
cognitive  tasks  (see  Richards  8  Siegler  1982)  These  findings  suggest  that  a  short-term 
decrement  in  certain  aspects  of  task  performance  may  be  a  necessary  consequence  of  the 
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early  forms  of  schematic  knowledge  that  develop  in  the  course  of  acquiring  expertise  in  a 
variety  of  content  domains 

The  importance  rules  developed  by  novices  have  important  consequences  for  what 
novices  learn  from  texts  Previous  research  has  found  that  novices  spend  more  time  on 
information  categories  judged  as  important  when  reading  physics  passages,  recall  more 
information  from  categories  fudged  as  important,  and  include  more  information  from  these 
categories  in  their  summaries  of  physics  texts  (Dee-Lucas  &  Larkin.  1986.  1987).  Thus 
these  information-category  rules  appear  to  influence  novice  readers'  attentional  processes 
during  reading,  as  well  as  the  macrostructure  they  develop  for  physics  texts.  These  rules 
therefore  determine  in  part  what  novices  learn  from  these  texts  The  specific  findings  from 
the  current  research  suggest  that  novices  may  be  missing  important  facts  and  quantitative 
relations,  and  attending  to  some  less  important  definitions  and  equations  when  studying 
these  types  of  texts 

Additionally,  this  research  suggests  that  novice  rules  may  influence  novice  problem¬ 
solving  performance,  as  well  as  what  is  retained  The  ability  to  select  from  a  text  the 
relevant  quantitative  relations  is  crucial  for  skillful  problem-solving.  The  finding  that  novices 
consider  equations  to  be  more  important  than  verbal  formulae  suggests  that  novices  may  not 
readily  recognize  quantitative  relations  as  being  relevant  in  a  problem-solving  situation  when 
this  information  is  not  presented  in  equational  form.  Similarly,  novices  may  be  more  likely  to 
attend  to  irrelevant  quantitative  relations  when  they  are  presented  as  equations.  This 
inability  to  pinpoint  problem-relevant  information  in  a  text  would  decrease  the  efficiency  (and 
possibly  accuracy)  with  which  novices  solve  physics  problems.  Thus  the  nature  of  "ovio 
rules  has  important  implications  for  novice  problem-solving  performance  as  well  as  text  recall 

These  findings  suggest  that  certain  variations  in  wording  or  form  which  do  not  affect 
the  reading  of  experts  or  domain-naive  readers  will  have  essentially  a  "signaling"  effect  on 
novice  attention  and  recall  Signaling  involves  the  use  of  non-content  words,  such  as  "more 
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importantly."  "note  that,"  etc  to  emphasise  particular  text  information  Signaling  has  been 
found  to  alter  readers'  attention,  as  indicated  by  differences  in  recall  patterns  for  signaled 
and  unsignaled  texts  (Loman  &  Mayer.  1983:  Meyer.  1983).  The  results  of  the  current  study 
show  that  there  are  particular  alternative  wordings  (eg.,  "is  defined  as"  vs.  represented 
as")  and  presentation  styles  (e  g  .  symbolic  vs.  verbal)  which  do  not  function  as  signaling 

devices  for  most  readers,  but  have  a  signaling  effect  for  novices  because  they  reflect  the 

categorization  scheme  used  by  novices  to  assess  importance  Therefore  authors  can 

unintentionally  signal  particular  information  as  important  through  presentation  selections  of  this 
type  if  they  are  unaware  of  novice  preconceptions  regarding  the  text  content  domain. 

On  the  other  hand,  an  author  can  use  knowledge  of  novice  importance  rules  in 

conjunction  with  signaling  techniques  to  guide  readers'  attention  to  the  appropriate  text 

content  Writers  can  use  signaling  devices  to  emphasize  important  content  within  the 
sentence-type  categories  that  novices  consider  unimportant,  and  de-emphasize  less  critical 
information  in  the  categories  that  novices  judge  as  important  Meyer  (1975)  describes  four 
types  of  signaling  techniques:  (1)  providing  cues  as  to  the  text  structure,  as  In  "the 
problem  is  the  solution  is."  or  "first,  second  ,  third:"  (2)  paraphrasing  important  text  content 
before  it  is  presented,  such  as  "the  important  points  in  the  following  discussion  are...:’'  (3) 
summary  statements  of  key  ideas  presented  after  the  relevant  text:  and  (4)  pointer  words 

emphasizing  specific  statements  in  the  texts,  as  in  "more  importantly,"  "it  is  notable  that.” 

"unfortunately,"  etc.  These  devices  can  be  used  independently  of  the  overall  hierarchical 

structure  of  the  passage  to  emphasize  specific  content  within  each  level,  a  technique  that 
Meyer  (1983)  terms  a  "differential  emphasis  plan”  These  techniques  have  been  found  to 
be  effective  in  altering  recall  patterns  of  readers  of  varying  backgrounds  and  abilities  (Loman 
&  Mayer.  1983:  Meyer.  1983) 

Signaling  devices  can  be  used  by  writers  of  physics  texts  to  help  guide  novices 
attention  to  the  important  content  For  example,  the  current  study  indicates  that  definitions 
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are  a  type  of  content  that  is  particularly  salient  to  novices.  Therefore,  it  would  be  helpful 
to  novices  if  writers  would  differentiate  the  less  important  definitions  from  those  that  are 
central  to  the  text.  This  could  be  done  by  including  structural  cues  or  preliminary 
summaries  that  emphasize  the  main  topics  or  content  that  the  author  wants  the  reader  to 
abstract  from  the  text,  and  thus  in  effect  "de-emphasize"  unimportant  definitions.  This 

would  help  novices  distinguish  between  the  definitions  that  are  to  be  learned  (i . e . .  the  main 
points)  and  those  that  are  there  to  simply  aid  in  comprehension  or  elaborate  on  the 

important  points.  Facts,  on  the  other  hand,  are  not  particularly  salient  for  novices,  and 
novices  may  be  missing  important  content  of  this  type  (Dee-Lucas  &  Larkin.  1986).  In  this 
case,  the  use  of  underlining  and  pointer  words  (e  g.,  note  that,  it  is  important  to  undt  stand, 
this  leads  to  the  important  conclusion  that,  etc.)  could  be  used  to  draw  attention  to 
important  facts. 

Another  technique  that  could  be  used  to  help  novices  distinguish  between  important 

and  unimportant  information  is  to  teach  novices  a  general  learning  schema  that  can  be 
applied  to  texts  to  assess  importance.  This  approach  to  manipulating  attention  and  learning 
is  "strategy-based"  rather  than  "text-based."  The  ability  of  students  to  acquire  general 
skills  for  learning  from  scientific  texts  has  been  demonstrated  by  Larkin  and  Reif  (1976). 

They  taught  students  a  skill  for  understanding  quantitative  relations  in  physics  texts  by  having 
students  work  through  a  series  of  training  materials  that  required  them  to  read  physics  texts 
and  answer  a  prespecified  set  of  questions.  The  subjects  were  able  to  learn  to  use  this 
question  set  as  a  strategy  for  acquiring  an  understanding  of  new  relations,  and  performed 
appreciably  better  on  tests  of  understanding  than  subjects  who  had  not  acoi'"*  ■'  I1'1' 
learning  strategy.  In  the  case  of  the  current  study,  the  goal  of  the  learning  schema  would 
be  to  help  readers  distinguish  between  important  and  unimportant  information  within 
categories  typically  viewed  as  unimportant,  such  as  facts  For  example.  Larkin  and  Reif 
were  able  to  specify  a  subset  of  facts  (i  e. .  those  dealing  with  units  and  typical  magnitudes) 
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that  are  particularly  important  for  understanding  quantitative  relations.  This  type  of  schema 
could  be  used  by  novices  in  reading  scientific  texts  to  aid  in  identification  of  the  important 
content  within  categories  of  information  that  they  generally  assume  are  unimportant.  This 
schema  would  hold  true  for  a  broad  content  area  within  the  domain  of  physics,  but  would 
most  likely  vary  to  some  extent  with  subfields  of  study  in  physics. 

The  present  study  indicates  that  novice  readers  can  be  sensitive  to  very  minor  changes 
in  wording  at  the  sentence  level  in  a  text,  in  that  these  changes  can  be  relevant  to 

distinctions  made  in  their  content  schema  for  the  text  domain.  This  suggests  that  writers 

need  to  be  aware  of  the  manner  of  presentation  of  information  at  a  fairly  local  level  in  a 

text.  In  particular,  experts  writing  for  a  novice  audience  will  be  most  effective  in  enhancing 

learning  of  the  important  text  content  if  they  use  text-based  indicators  of  importance  in 
conjunction  with  knowledge  of  their  audiences'  content  schema.  In  this  way.  writers  can  use 
techniques  such  as  signaling  to  provide  novices  with  clear  signals  of  importance  which  will 
help  them  distinguish  between  important  and  less  critical  content. 
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Table  1:  Sample  target  sentences 


(a)  Examples  of  definition  and  fact  versions  (Experiment  1). 


t.  Absolute  pressure  is  defined  as  simply  the  actual  pressure  at  a  point. 

Absolute  pressure  is  simply  the  actual  pressure  at  a  point. 

2.  Specific  gravity  is  defined  as  the  ratio  of  the  density  of  a  substance  to  the 
density  of  water. 

Specific  gravity  is  indicated  by  the  ratio  of  the  density  of  a  substance  to  the 

density  of  water. 

3.  In  terms  of  this  notation,  the  work  A  U  done  by  a  force  F  in  moving  an  object 
through  a  displacement  Ar  is  defined  as 

A  IV  =  F  •  Ar  =  /•'  I A  >'  cos-  fi  I 

In  terms  of  this  notation,  the  work  a  IV  done  by  a  force  F  in  moving  an  object 
through  a  displacement  Ar  is  represented  as 

A  IV  =  F  •  Ar  =  /■  (A  >'  cos  H  I 

4.  The  unit  typically  used  for  measuring  work,  the  joule,  is  defined  as  the  work 

done  by  a  unit  force  (one  newton)  acting  on  a  unit  distance  (one  meter), 

The  unit  typically  used  for  measuring  work,  the  joule,  indicates  the  amount  of 

work  done  by  a  unit  force  (one  newton)  acting  on  a  unit  distance  (one  meter). 

5.  Pressure  is  defined  as  the  magnitude  of  a  fluid  force  divided  by  the  area  of  the 
surface  on  which  it  acts 

Pressure  can  be  calculated  by  dividing  the  magnitude  of  a  fluid  force  by  the 

area  of  the  surface  on  which  it  acts 


(b)  Examples  of  equation  and  verbal  versions  (Experiment  2) 

1.  Kinetic  energy  K  is  equal  to  the  produce  of  one-half  the  mass  m  of  the  particle 
times  the  square  r~  of  its  speed 

Kinetic  energy  is 
K  =  1/2  mr2 

where  m  is  the  mass  of  the  particle  and  r  is  its  speed 


‘-TXT 


Novice  Rules 


2.  The  velocity  v  of  an  object  is  equal  to  the  rate  at  which  its  position  changes 
with  time,  or  the  displacement  Ar  divided  by  the  corresponding  time  interval  A  t. 

The  velocity  of  an  object  is  equal  to  the  rate  at  which  its  position  changes  with 
time  or 

v  =  Ar  /  At 

where  Ar  is  the  displacement  and  a  /  is  the  corresponding  time  interval. 

3.  The  gauge  pressure  /*  at  a  point  in  a  fluid  is  equal  to  the  difference  between 
the  pressure  /'  at  that  point  and  the  atmospheric  pressure  Panno<;- 

The  gauge  pressure  at  a  point  in  a  fluid  is 
/»=/’-  P 

a  1 1  nnw's 

where  /’  is  the  pressure  at  that  point  and  PtUmo,  is  the  atmospheric  pressure. 

4  Density  p  is  equal  to  the  mass  m  of  a  portion  of  material  divided  by  the  volume 

V  of  that  portion 

Density  is 

p  =  in  i  r 

where  in  is  the  mass  of  a  portion  of  material  and  V  is  the  volume  of  that 

portion. 

5  Specify  gravity  .S'  is  equal  to  the  density  p  s  of  a  substance  divided  by  the 

density  p  of  water. 

Specific  gravity  is 
•S'  =  /»,  1  Pu 

where  pK  is  the  density  of  a  substance  and  p  (  is  the  density  of  water. 


V  .»  A  ■>  A  •  .1  .N  X.V 
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Table  2:  Parameter  estimates,  standard  errors,  and  ratio  of  estimates  to 
standard  errors  for  a  loglinear  model  of  the  ratings  data  from  the 
experimental  groups  (Experiment  1).  Asterisks  indicate  coefficients  that 
differ  from  zero  by  more  than  2  standard  errors. 

Ratio: 

Effect  Coeff.  St.  Error  C oeff./St.  Error 

(a)Group:  novice  estimates  (expert  estimates  are  opposite) 


Rate  1: 

045 

.066 

.69 

Rate  2: 

.074 

.075 

.98 

Rate  3: 

.094 

090 

1.04 

Rate  4: 

.019 

.118 

.16 

Rate  5: 

-  232 

.182 

-1.27 

(b)Type:  definition 

estimates  (fact  estimates  are  opposite) 

Rate  1: 

.073 

.067 

1.10 

Rate  2 

.082 

.075 

1.10 

Rate  3: 

-  224* 

.090 

-2.48 

Rate  4: 

064 

115 

.56 

Rate  5: 

.005 

.192 

.03 

(c)Type  x  Group: 

definition  estimates 

(fact  estimates  are 

opposite)1 

Rate  1: 

Novice 

(  +  ) 

.133* 

066 

2.01 

Expert 

(-) 

Rate  2: 

Novice 

(-) 

.098 

.075 

1.30 

Expert 

(  +  ) 

Rate  3: 

Novice 

(  +  ) 

.003 

081 

.04 

Expert 

(-) 

Rate  4: 

Novice 

(  +  ) 

.014 

117 

.12 

Expert  (-) 

Rate  5. 

Novice  (-)  052  180  .29 

Expert  ( + ) 

'Parameter  estimates  indicate  the  size  of  the  effect:  positive  and  negative  symbols 
indicate  the  direction  of  the  effect  for  each  group 


'  K~J<T  WA'A'A’ 
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(d)Level 

Level  1 

Rate 

1 

Rate 

2 

Rate 

3 

Rate 

4 

Rate 

5 

Level  2 

Rate 

1 

Rate 

2 

Rate 

3 

Rate 

4 

Rate 

5 

Level  3: 

Rate 

1 

Rate 

2 

Rate 

3 

Rate 

4 

Rate 

5 
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Table  3:  Parameter  estimates,  standard  errors,  and  ratios  of  estimates 

to  standard  errors  for  a  loglinear  model  of  the  ratings  data  from  the 
control  groups  (Experiment  1).  Asterisks  indicate  coefficients  that  differ 
from  zero  by  more  than  2  standard  errors. 

Ratio: 

Effect  Coeff.  St.  Error  Coeff./St.  Error 

(a)Group:  urtdergrad  estimates  (grad  estimates  are  opposite) 


Rate  1: 

323* 

065 

4.98 

Rate  2: 

.083 

.074 

1.12 

Rate  3: 

-.070 

.089 

-.79 

Rate  4: 

-.323* 

.117 

-2.76 

Rate  5: 

-.010 

.172 

-.06 

(b)Level 

Level  1: 

Rate  1 

695* 

.105 

6.62 

Rate  2 

.141 

.119 

1.18 

Rate  3 

-.302 

156 

-1.94 

Rate  4 

-.492 - 

.220 

-2.24 

Rate  5 

-.042 

.282 

-.15 

Level  2. 

Rate  1 

.128 

102 

1.25 

Rate  2 

.013 

.118 

.11 

Rate  3 

.133 

.137 

.97 

Rate  4 

-.047 

.188 

-.25 

Rate  5 

-.226 

.281 

-.80 

Level  3: 

Rate  1 

-.822* 

.092 

-8.92 

Rate  2 

-.154 

099 

-1.56 

Rate  3 

.169 

.119 

1.42 

Rate  4 

539* 

155 

3.48 

Rate  5 


268 


226 


1.19 
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Table  4:  Parameter  estimates,  standard  errors,  and  ratios  of  estimates 

to  standard  errors  for  a  loqistic  regression  model  of  the  sentence 
selection  data  from  the  experimental  groups  (Experiment  1). 


Effect 

Coeff. 

St.  Error 

Coeff. /St.  Error 

Type: 

-.414 

.099 

-4.18 

Group: 

-  186 

.099 

-1.88 

Type  x  Group: 

.236 

.099 

2  38 

Definition  (-) 

Fact  (  +  ) 

Level: 

-1.016 

.088 

-1 1.49 

,‘v’\  v'A 
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Table  5:  Parameter  estimates,  standard  errors,  and  ratios  of  estimates  to  standard 
errors  for  a  logistic  regression  model  of  the  sentence  selection  data  from  the 

control  groups  (Experiment  1). 


Effect 

Coeff. 

St.  Error 

Ratio: 

Coeff. /St.  Error 

Type: 

-220 

.098 

-2.24 

Type  x  Level: 

Level  1 : 

-.448 

139 

-3.22 

Level  2: 

014 

139 

10 

Level  3: 

462 

140 

3.30 

Group: 

-.590 

098 

-6  02 

Level: 

Level  1-2  diff 

476 

139 

3.42 

Level  2-3  diff: 

1.316 

.140 

9.40 
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Table  6:  Parameter  estimates,  standard  errors,  and  ratio  of  estimates  to 
standard  errors  for  a  loglinear  model  of  the  ratings  data  from  the 
experimental  groups  (Experiment  2).  Asterisks  indicate  coefficients  that 
differ  from  zero  by  more  than  2  standard  errors 

Ratio: 

Effect  Coeff.  St.  Error  Coeff./St.  Error 

(a)Group:  novice  estimates  (expert  estimates  are  opposite) 


Rate  1: 

-.016 

081 

-.20 

Rate  2: 

.116 

098 

1.19 

Rate  3: 

.003 

.125 

.02 

Rate  4: 

-.187 

.182 

-1.03 

Rate  5: 

.085 

.205 

.42 

(b)Type:  equation 

estimates  (verbal  estimates 

are  opposite) 

Rate  1: 

.297- 

.083 

3.60 

Rate  2: 

-.154 

.098 

-1.58 

Rate  3: 

-  031 

.119 

-.26 

Rate  4. 

-  195 

183 

-1.07 

Rate  5 

.083 

.205 

.40 

(c)Type  x  Group: 

equation  estimates  (verbal 

estimates  are 

opposite)1 

Rate  1: 

Novice 

(  +  ) 

236' 

.083 

2.86 

Expert 

(-) 

Rate  2. 

Novice 

(  +  ) 

035 

.098 

.36 

Expert 

(-) 

Rate  3: 

Novice 

(  +  ) 

.027 

120 

23 

Expert 

(-) 

Rate  4 

Novice 

(-) 

-.256 

182 

1  40 

Expert 

(  +  ) 

Rate  5: 

Novice 

(-) 

-.042 

205 

-  21 

Expert  (  * ) 

'Parameter  estimates  indicate  the  size  of  the  effect  positive  and  negative  symbols 
indicate  the  direction  of  the  effect  for  each  group 


.’■’/  VV'.,V.V.V.V.V.V.V.  V.  r. 


Rate  1 

1  065- 

164 

6.50 

Rate  2 

-.216 

.200 

-1.08 

Rate  3 

-.233 

249 

-.94 

Rate  4 

-.461 

417 

-1.11 

Rate  5 

-.155 

423 

-.37 

Level  2: 

Rate  1 

-.087 

135 

-.64 

Rate  2 

352- 

154 

2  29 

Rate  3 

.158 

.191 

83 

Rate  4 

-.252 

.323 

78 

Rate  5 

-  170 

.348 

-.49 

Level  3: 

Rate  1 

-.978' 

.122 

-8.01 

Rate  2 

-.136 

139 

-  98 

Rate  3 

.075 

.172 

.44 

Rate  4 

713" 

.264 

2.70 

Rate  5 

325 

.291 

1.12 

46 
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Table  7:  Parameter  estimates,  standard  errors,  and  ratios  of  estimates  to  standard 

errors  for  a  loglinear  model  of  the  ratings  data  from  the  control  groups 
(Experiment  2).  Asterisks  indicate  coefficients  that  differ  from  zero  by  more  than 


standard  errors 

Effect 

Coeff. 

St. 

Error 

Ratio: 

Coeff. /St.  Error 

(a)Group: 

undergrad 

estimates  (grad 

estimates 

are  opposite) 

Rate  1: 

.280' 

.083 

3.39 

Rate  2: 

.018 

.099 

.18 

Rate  3: 

-.301  ‘ 

115 

-2  63 

Rate  4: 

-.314' 

135 

2.33 

Rate  5: 

-.312 

.240 

-1.30 

(b)Level 

Level  1 

Rate 

1 

.482' 

.133 

3  62 

Rate 

2 

036 

162 

22 

Rate 

3 

097 

.172 

.56 

Rate 

4 

-.213 

.227 

-  94 

Rate 

5 

-.402 

.417 

-.97 

Level  2 

Rate 

1 

-.001 

.250 

-.01 

Rate 

2 

.067 

143 

.47 

Rate 

3 

-.100 

157 

-.64 

Rate 

4 

.052 

193 

.27 

Rate 

5 

.019 

352 

-.05 

Level  3 

Rate 

1 

-482' 

.110 

-4.37 

Rate 

2 

-  102 

133 

-.77 

Rate 

3 

003 

158 

.02 

Rat" 

4 

161 

179 

90 

Rate 

5 

420 

303 

1.39 

AA ////■' z  i 
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(c)Type:  equation  estimates  (verbal  estimates  are  opposite) 


Rate  1 

150 

078 

1  92 

Rate  2 

-.181 

098 

-1.84 

Rate  3 

008 

103 

.08 

Rate  4 

104 

131 

79 

Rate  5 

-  082 

.220 

-.37 

^V/VvV/vV-.-'/V/v'* 


>V>V.VM-'>VvVj.V.WjiW, 


'Xu*'*  *  »  *  •  -*>''»  *  •  . v  w  •  .  *  '•  ,  «  .  -  * 

v*  •  ■  *  '  *V-  V' 
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Table  8:  Parameter  estimates,  standard  errors,  and  ratios  of  estimates 

to  standard  errors  for  a  logistic  regression  model  of  the  sentence 
selection  data  from  the  experimental  groups  (Experiment  2). 


Effect 

Coe  ff. 

St.  Error 

Ratio 

Coeff./St.  Error 

Type: 

-1.100 

137 

-8.03 

Group: 

-.530 

.134 

-3.96 

Type  x  Group: 

.842 

.136 

6.19 

Verbal  ( + ) 

Equation  (-) 

Level: 

-1.239 

126 

-9.82 
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Table  9:  Parameter  estimates,  standard  errors,  and  ratios  of  estimates  to  standard 
errors  for  a  logistic  regression  model  of  the  sentence  selection  data  from  the 

control  groups  (Experiment  2). 

Ratio: 


Effect 

Coeff. 

St.  Error 

Coeff. /St.  Error 

Type: 

-.544 

120 

-4.52 

Type  x  Level: 

Level  1 : 

.262 

176 

1  49 

Level  2: 

-.652 

.171 

-3.82 

Level  3: 

.390 

.165 

2.36 

Group: 

-.756 

.120 

-6.28 

Level: 

Level  1-2  diff: 

.611 

.174 

3.51 

Level  2-3  diff: 

.530 

.168 

3  15 
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Figure  1: 


Predicted  mean  ratings  for  the  type  x  group  interaction  in  the 
ratings  data  (Experiment  1). 


(a)  Experimental  Groups 


ibi  Control  Groups 
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Figure 


2:  Predicted  proportion  of  target  sentences  selected  as  important  for 

the  type  x  group  Interaction  in  the  sentence  selection  data.  (Experiment  i) 


(a)  Experimental  Groups 


(bt  Control  GrouDS 
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Figure  4: 


Predicted  mean  ratings  for  the  type  x  group  interaction  in  the 
ratings  data  (Experiment  2). 


Group 

(a)  Experimental  Groups 


Group 


(b)  Control  GroubS 
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Figure  6:  Predicted  proportion  of  target  sentences  selected  as  important  for 

the  type  x  level  interaction  in  the  sentence  selection  data  for  the  control 

groups  (Experiment  2). 
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